🐟Fish Classification🔎
🐠(Prediction at the end)🔮

A Large-Scale Dataset for Segmentation and Classification
Authors: O. Ulucan, D. Karakaya, M. Turkan Department of Electrical and Electronics Engineering, Izmir University of Economics, Izmir, Turkey Corresponding author: M. Turkan Contact Information: mehmet.turkan@ieu.edu.tr
Paper : A Large-Scale Dataset for Fish Segmentation and Classification General Introduction
This dataset contains 9 different seafood types collected from a supermarket in Izmir, Turkey for a university-industry collaboration project at Izmir University of Economics, and this work was published in ASYU 2020. The dataset includes gilt head bream, red sea bream, sea bass, red mullet, horse mackerel, black sea sprat, striped red mullet, trout, shrimp image samples.
Purpose of the work
This dataset was collected in order to carry out segmentation, feature extraction, and classification tasks and compare the common segmentation, feature extraction, and classification algorithms (Semantic Segmentation, Convolutional Neural Networks, Bag of Features). All of the experiment results prove the usability of our dataset for purposes mentioned above.
Data Gathering Equipment and Data Augmentation
Images were collected via 2 different cameras, Kodak Easyshare Z650 and Samsung ST60. Therefore, the resolution of the images are 2832 x 2128, 1024 x 768, respectively.
Before the segmentation, feature extraction, and classification process, the dataset was resized to 590 x 445 by preserving the aspect ratio. After resizing the images, all labels in the dataset were augmented (by flipping and rotating).
At the end of the augmentation process, the number of total images for each class became 2000; 1000 for the RGB fish images and 1000 for their pair-wise ground truth labels.
Description of the dataset
The dataset contains 9 different seafood types. For each class, there are 1000 augmented images and their pair-wise augmented ground truths. Each class can be found in the "Fish_Dataset" file with their ground truth labels. All images for each class are ordered from "00000.png" to "01000.png".
For example, if you want to access the ground truth images of the shrimp in the dataset, the order should be followed is "Fish->Shrimp->Shrimp GT".
import numpy as np
import pandas as pd
import os
from pathlib import Path
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.decomposition import PCA
from sklearn.model_selection import train_test_split
from sklearn.metrics import f1_score, accuracy_score, confusion_matrix, classification_report, roc_curve
from sklearn.model_selection import learning_curve, cross_val_score, GridSearchCV
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier
from sklearn.svm import SVC
from sklearn.pipeline import make_pipeline
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import RobustScaler,StandardScaler,MinMaxScaler
import warnings
warnings.filterwarnings('ignore')
image_dir = Path('../input/a-large-scale-fish-dataset/Fish_Dataset/Fish_Dataset')
# Get filepaths and labels
filepaths = list(image_dir.glob(r'**/*.png'))
labels = list(map(lambda x: os.path.split(os.path.split(x)[0])[1], filepaths))
filepaths = pd.Series(filepaths, name='Filepaths').astype(str)
labels = pd.Series(labels, name='Labels')
# Concatenate filepaths and labels
image_df = pd.concat([filepaths, labels], axis=1)
# Drop GT images
image_df = image_df[image_df['Labels'].apply(lambda x: x[-2:] != 'GT')]
# Shuffle the DataFrame and reset index
image_df = image_df.sample(frac=1).reset_index(drop = True)
# Show the result
image_df.head()
| Filepaths | Labels | |
|---|---|---|
| 0 | ../input/a-large-scale-fish-dataset/Fish_Datas... | Red Mullet |
| 1 | ../input/a-large-scale-fish-dataset/Fish_Datas... | Trout |
| 2 | ../input/a-large-scale-fish-dataset/Fish_Datas... | Sea Bass |
| 3 | ../input/a-large-scale-fish-dataset/Fish_Datas... | Red Sea Bream |
| 4 | ../input/a-large-scale-fish-dataset/Fish_Datas... | Trout |
#Displaying a subsample of the dataset
fig, axes = plt.subplots(nrows=3, ncols=5, figsize=(15,10), subplot_kw={'xticks':[], 'yticks':[]})
for i, ax in enumerate(axes.flat):
ax.imshow(plt.imread(image_df.Filepaths[i]))
ax.set_title(image_df.Labels[i])
plt.show()
create_report(image_df)
Dataset Statistics
| Number of Variables | 2 |
|---|---|
| Number of Rows | 9000 |
| Missing Cells | 0 |
| Missing Cells (%) | 0.0% |
| Duplicate Rows | 0 |
| Duplicate Rows (%) | 0.0% |
| Total Size in Memory | 1.9 MB |
| Average Row Size in Memory | 222.0 B |
| Variable Types |
|
Dataset Insights
| Filepaths has a high cardinality: 9000 distinct values | High Cardinality |
|---|---|
| Filepaths has all distinct values | Unique |
Filepaths
categorical
| Approximate Distinct Count | 9000 |
|---|---|
| Approximate Unique (%) | 100.0% |
| Missing | 0 |
| Missing (%) | 0.0% |
| Memory Size | 1.4 MB |
Length
| Mean | 96.3333 |
|---|---|
| Standard Deviation | 8.6415 |
| Median | 99 |
| Minimum | 83 |
| Maximum | 109 |
Sample
| 1st row | ../input/a-large-s... |
|---|---|
| 2nd row | ../input/a-large-s... |
| 3rd row | ../input/a-large-s... |
| 4th row | ../input/a-large-s... |
| 5th row | ../input/a-large-s... |
Letter
| Count | 656000 |
|---|---|
| Lowercase Letter | 580000 |
| Space Separator | 20000 |
| Uppercase Letter | 76000 |
| Dash Punctuation | 38000 |
| Decimal Number | 45000 |
- Filepaths contains many words: 7015 words
- The largest value (sea) is over 2.0 times larger than the second largest value (red)
Labels
categorical
| Approximate Distinct Count | 9 |
|---|---|
| Approximate Unique (%) | 0.1% |
| Missing | 0 |
| Missing (%) | 0.0% |
| Memory Size | 673.8 KB |
Length
| Mean | 11.6667 |
|---|---|
| Standard Deviation | 4.3207 |
| Median | 13 |
| Minimum | 5 |
| Maximum | 18 |
Sample
| 1st row | Red Mullet |
|---|---|
| 2nd row | Trout |
| 3rd row | Sea Bass |
| 4th row | Red Sea Bream |
| 5th row | Trout |
Letter
| Count | 94000 |
|---|---|
| Lowercase Letter | 74000 |
| Space Separator | 10000 |
| Uppercase Letter | 20000 |
| Dash Punctuation | 1000 |
| Decimal Number | 0 |
Modelling¶
Creating the model¶
import tensorflow as tf
# Separate in train and test data
train_df, test_df = train_test_split(image_df, train_size=0.85, shuffle=True, random_state=1)
train_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input,
validation_split=0.15
)
test_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input
)
train_images = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Filepaths',
y_col='Labels',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=64,
shuffle=True,
seed=42,
subset='training'
)
Found 6503 validated image filenames belonging to 9 classes.
val_images = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Filepaths',
y_col='Labels',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=64,
shuffle=True,
seed=42,
subset='validation'
)
Found 1147 validated image filenames belonging to 9 classes.
test_images = test_generator.flow_from_dataframe(
dataframe=test_df,
x_col='Filepaths',
y_col='Labels',
target_size=(224, 224),
color_mode='rgb',
class_mode='categorical',
batch_size=64,
shuffle=False
)
Found 1350 validated image filenames belonging to 9 classes.
from keras.models import Sequential, Model
from keras.layers import Dense, Flatten, Dropout, GlobalAveragePooling2D
input_shape = (224, 224, 3)
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=input_shape ),
tf.keras.layers.MaxPool2D(pool_size = (2,2)),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPool2D(pool_size = (2,2)),
tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
tf.keras.layers.MaxPool2D(pool_size = (2,2)),
tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
tf.keras.layers.MaxPool2D(pool_size = (2,2)),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(9, activation='softmax')
])
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 222, 222, 64) 1792 _________________________________________________________________ max_pooling2d (MaxPooling2D) (None, 111, 111, 64) 0 _________________________________________________________________ conv2d_1 (Conv2D) (None, 109, 109, 64) 36928 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 54, 54, 64) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 52, 52, 32) 18464 _________________________________________________________________ max_pooling2d_2 (MaxPooling2 (None, 26, 26, 32) 0 _________________________________________________________________ conv2d_3 (Conv2D) (None, 24, 24, 32) 9248 _________________________________________________________________ max_pooling2d_3 (MaxPooling2 (None, 12, 12, 32) 0 _________________________________________________________________ flatten (Flatten) (None, 4608) 0 _________________________________________________________________ dense (Dense) (None, 512) 2359808 _________________________________________________________________ dense_1 (Dense) (None, 128) 65664 _________________________________________________________________ dropout (Dropout) (None, 128) 0 _________________________________________________________________ dense_2 (Dense) (None, 128) 16512 _________________________________________________________________ dropout_1 (Dropout) (None, 128) 0 _________________________________________________________________ dense_3 (Dense) (None, 9) 1161 ================================================================= Total params: 2,509,577 Trainable params: 2,509,577 Non-trainable params: 0 _________________________________________________________________
from tensorflow.keras.optimizers import Adam
model.compile(optimizer="adam", loss='categorical_crossentropy', metrics=["accuracy"])
callback = tf.keras.callbacks.EarlyStopping(monitor='accuracy', patience=4)
history = model.fit(train_images, validation_data=val_images, epochs=6, callbacks=callback)
Epoch 1/6 102/102 [==============================] - 177s 2s/step - loss: 1.9274 - accuracy: 0.2592 - val_loss: 0.8328 - val_accuracy: 0.7018 Epoch 2/6 102/102 [==============================] - 77s 755ms/step - loss: 0.7174 - accuracy: 0.7444 - val_loss: 0.4143 - val_accuracy: 0.8527 Epoch 3/6 102/102 [==============================] - 77s 760ms/step - loss: 0.2891 - accuracy: 0.9041 - val_loss: 0.2049 - val_accuracy: 0.9294 Epoch 4/6 102/102 [==============================] - 77s 755ms/step - loss: 0.1519 - accuracy: 0.9503 - val_loss: 0.1049 - val_accuracy: 0.9608 Epoch 5/6 102/102 [==============================] - 77s 755ms/step - loss: 0.0732 - accuracy: 0.9751 - val_loss: 0.1196 - val_accuracy: 0.9616 Epoch 6/6 102/102 [==============================] - 77s 752ms/step - loss: 0.0810 - accuracy: 0.9771 - val_loss: 0.1206 - val_accuracy: 0.9599
model.save('1rst-model.h5')
Visualize the filters¶
from tensorflow.keras.preprocessing.image import img_to_array, load_img
img_path=test_df['Filepaths'][5221]
# Define a new Model, Input= image
# Output= intermediate representations for all layers in the
# previous model after the first.
successive_outputs = [layer.output for layer in model.layers[1:]]
#visualization_model = Model(img_input, successive_outputs)
visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)
#Load the input image
img = load_img(img_path, target_size=(224, 224))
# Convert ht image to Array of dimension (150,150,3)
x = img_to_array(img)
x = x.reshape((1,) + x.shape)
# Rescale by 1/255
x /= 255.0
# Let's run input image through our vislauization network
# to obtain all intermediate representations for the image.
successive_feature_maps = visualization_model.predict(x)
# Retrieve are the names of the layers, so can have them as part of our plot
layer_names = [layer.name for layer in model.layers]
for layer_name, feature_map in zip(layer_names, successive_feature_maps):
print(feature_map.shape)
if len(feature_map.shape) == 4:
# Plot Feature maps for the conv / maxpool layers, not the fully-connected layers
n_features = feature_map.shape[-1] # number of features in the feature map
size = feature_map.shape[ 1] # feature map shape (1, size, size, n_features)
# We will tile our images in this matrix
display_grid = np.zeros((size, size * n_features))
# Postprocess the feature to be visually palatable
for i in range(n_features):
x = feature_map[0, :, :, i]
x -= x.mean()
x /= x.std ()
x *= 64
x += 128
x = np.clip(x, 0, 255).astype('uint8')
# Tile each filter into a horizontal grid
display_grid[:, i * size : (i + 1) * size] = x
# Display the grid
scale = 20. / n_features
plt.figure( figsize=(scale * n_features, scale) )
plt.title ( layer_name )
plt.grid ( False )
plt.imshow( display_grid, aspect='auto', cmap='viridis' )
(1, 111, 111, 64) (1, 109, 109, 64) (1, 54, 54, 64) (1, 52, 52, 32) (1, 26, 26, 32) (1, 24, 24, 32) (1, 12, 12, 32) (1, 4608) (1, 512) (1, 128) (1, 128) (1, 128) (1, 128) (1, 9)
Performances¶
accuracy = history.history['accuracy']
val_accuracy = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']
plt.figure(figsize=(15,10))
plt.subplot(2, 2, 1)
plt.plot(accuracy, label = "Training accuracy")
plt.plot(val_accuracy, label="Validation accuracy")
plt.legend()
plt.title("Training vs validation accuracy")
plt.subplot(2,2,2)
plt.plot(loss, label = "Training loss")
plt.plot(val_loss, label="Validation loss")
plt.legend()
plt.title("Training vs validation loss")
plt.show()
pred = model.predict(test_images)
pred = np.argmax(pred, axis=1)
labels = train_images.class_indices
labels = dict((v,k) for k, v in labels.items())
print(labels)
print(pred)
{0: 'Black Sea Sprat', 1: 'Gilt-Head Bream', 2: 'Hourse Mackerel', 3: 'Red Mullet', 4: 'Red Sea Bream', 5: 'Sea Bass', 6: 'Shrimp', 7: 'Striped Red Mullet', 8: 'Trout'}
[0 0 6 ... 3 3 4]
y_pred = [labels[k] for k in pred]
print(classification_report(test_df.Labels, y_pred))
print('--------------------------------')
print(confusion_matrix(test_df.Labels, y_pred))
print('--------------------------------')
precision recall f1-score support
Black Sea Sprat 0.97 0.99 0.98 150
Gilt-Head Bream 0.92 0.99 0.95 169
Hourse Mackerel 0.98 0.99 0.98 135
Red Mullet 0.99 0.99 0.99 156
Red Sea Bream 0.99 0.92 0.95 159
Sea Bass 0.97 0.93 0.95 149
Shrimp 1.00 0.96 0.98 144
Striped Red Mullet 0.96 0.96 0.96 154
Trout 0.93 0.96 0.94 134
accuracy 0.97 1350
macro avg 0.97 0.97 0.97 1350
weighted avg 0.97 0.97 0.97 1350
--------------------------------
[[149 0 0 0 0 1 0 0 0]
[ 0 167 0 0 1 1 0 0 0]
[ 0 1 133 0 1 0 0 0 0]
[ 0 0 0 155 0 0 0 1 0]
[ 0 4 0 0 147 0 0 0 8]
[ 3 4 1 1 0 139 0 0 1]
[ 1 0 0 0 0 0 138 5 0]
[ 1 0 2 1 0 2 0 148 0]
[ 0 5 0 0 0 1 0 0 128]]
--------------------------------
model.evaluate(test_images)[1]
22/22 [==============================] - 14s 617ms/step - loss: 0.1145 - accuracy: 0.9659
0.965925931930542